Accuracy: The proportion of all predictions that were correct. It’s a great starting point but can be misleading for imbalanced datasets.
- Formula: \((TP + TN) / (TP + TN + FP + FN)\)
Balanced Accuracy: The average of Sensitivity and Specificity. It provides a more robust measure when classes are imbalanced by giving equal importance to both positive and negative classes.
- Formula: \((Sensitivity + Specificity) / 2\)
Sensitivity (Recall or True Positive Rate): Answers: “Of all the actual positive cases, how many did we correctly identify?” High sensitivity is crucial when you cannot afford to miss a positive case (e.g., disease screening).
- Formula: \(TP / (TP + FN)\)
Specificity (True Negative Rate): Answers: “Of all the actual negative cases, how many did we correctly identify?” High specificity is important when a false positive is very costly.
- Formula: \(TN / (TN + FP)\)
Precision (Positive Predictive Value): Answers: “Of all the cases we predicted as positive, how many were actually positive?” High precision matters when the cost of acting on a false positive is high.
- Formula: \(TP / (TP + FP)\)
Negative Predictive Value (NPV): Answers: “Of all the cases we predicted as negative, how many were actually negative?” It’s the counterpart to Precision.
- Formula: \(TN / (TN + FN)\)
F1 Score: The harmonic mean of Precision and Sensitivity. It provides a single score that balances the concerns of both metrics, making it useful when both false positives and false negatives are important to minimize.
- Formula: \(2 \times (Precision \times Sensitivity) / (Precision + Sensitivity)\)
Kappa (Cohen’s Kappa): Measures the agreement between the model’s predictions and the actual labels, corrected for the probability of agreement occurring by chance. A score of 1 indicates perfect agreement, 0 indicates agreement equivalent to random chance, and negative values indicate agreement worse than random.
Detection Rate: The proportion of the entire dataset that consists of true positives. It’s less commonly used than accuracy but gives a sense of how often the positive case is found in the population.
- Formula: \(TP / (TP + TN + FP + FN)\)